{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5HkhthoeyQnV"
      },
      "source": [
        "# Fueling Generative Content with Keyword Research\n",
        "\n",
        "Generative models have proven extremely useful in content idea generation. But they don’t take into account user search demand and trends. In this notebook, let’s see how we can solve that by adding keyword research into the equation.\n",
        "\n",
        "Read the accompanying [blog post here](https://txt.cohere.ai/generative-content-keyword-research/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nhMMerDZakc8"
      },
      "outputs": [],
      "source": [
        "# Install packages\n",
        "# TODO: upgrade to \"cohere>5\"",
"! pip install \"cohere<5\" topically > /dev/null"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "COGWkT7Z5HCw"
      },
      "outputs": [],
      "source": [
        "# Wrap output\n",
        "from IPython.display import HTML, display\n",
        "\n",
        "def set_css():\n",
        "  display(HTML('''\n",
        "  <style>\n",
        "    pre {\n",
        "        white-space: pre-wrap;\n",
        "    }\n",
        "  </style>\n",
        "  '''))\n",
        "get_ipython().events.register('pre_run_cell', set_css)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "4awUCb_3jw-v",
        "outputId": "ab8e8e13-99a8-47fc-8736-49a698476075"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import cohere\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from topically import Topically\n",
        "from sklearn.cluster import KMeans\n",
        "\n",
        "co = cohere.Client('api_key') # Add your Cohere API key here"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9YsKrQFsyZIE"
      },
      "source": [
        "# Step 1: Get a list of High-performing Keywords "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BbrQLsiwy5MH",
        "outputId": "f7656f35-dbb3-4111-fa29-dcbe7adf40c7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2023-03-30 11:08:34--  https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/remote_teams.csv\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 3764 (3.7K) [text/plain]\n",
            "Saving to: ‘remote_teams.csv’\n",
            "\n",
            "\rremote_teams.csv      0%[                    ]       0  --.-KB/s               \rremote_teams.csv    100%[===================>]   3.68K  --.-KB/s    in 0s      \n",
            "\n",
            "2023-03-30 11:08:34 (44.7 MB/s) - ‘remote_teams.csv’ saved [3764/3764]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Download the pre-created dataset (feel free to replace with your CSV file, containing two columns - \"keyword\" and \"volume\")\n",
        "!wget \"https://raw.githubusercontent.com/cohere-ai/notebooks/main/notebooks/data/remote_teams.csv\" -O remote_teams.csv"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "_JdLS_FVyjfr",
        "outputId": "211c8b07-da89-4269-a652-2c164d09338d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-b03e9ae0-113f-422a-8619-6b33c547ff49\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>keyword</th>\n",
              "      <th>volume</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>managing remote teams</td>\n",
              "      <td>1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>remote teams</td>\n",
              "      <td>390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>collaboration tools for remote teams</td>\n",
              "      <td>320</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>online games for remote teams</td>\n",
              "      <td>320</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>how to manage remote teams</td>\n",
              "      <td>260</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b03e9ae0-113f-422a-8619-6b33c547ff49')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b03e9ae0-113f-422a-8619-6b33c547ff49 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b03e9ae0-113f-422a-8619-6b33c547ff49');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                keyword  volume\n",
              "0                 managing remote teams    1000\n",
              "1                          remote teams     390\n",
              "2  collaboration tools for remote teams     320\n",
              "3         online games for remote teams     320\n",
              "4            how to manage remote teams     260"
            ]
          },
          "execution_count": 39,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a dataframe\n",
        "df = pd.read_csv('remote_teams.csv')\n",
        "df.columns = [\"keyword\",\"volume\"]\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whqG2M2Dylxu"
      },
      "source": [
        "# Step 2: Group the Keywords into Topics "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJbbahQA0BJb"
      },
      "source": [
        "### Embed the Keywords with co.embed"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "O7vz9gkXjGMI",
        "outputId": "d7f53d3d-e426-4433-a5be-1bb9c7d2b566"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def embed_text(text):\n",
        "  output = co.embed(\n",
        "                model='large',\n",
        "                texts=text)\n",
        "  return output.embeddings\n",
        "\n",
        "\n",
        "embeds = np.array(embed_text(df['keyword'].tolist()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GKnEI1E40Djn"
      },
      "source": [
        "### Cluster the Keywords into Topics with scikit-learn"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "jUAZueb0luKN",
        "outputId": "b05799e5-665d-4dcf-fb30-7eb7df1ed03f"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-5bba08db-2139-4423-b5f5-afad4d4d2a0a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>keyword</th>\n",
              "      <th>volume</th>\n",
              "      <th>topic</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>managing remote teams</td>\n",
              "      <td>1000</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>remote teams</td>\n",
              "      <td>390</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>collaboration tools for remote teams</td>\n",
              "      <td>320</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>online games for remote teams</td>\n",
              "      <td>320</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>how to manage remote teams</td>\n",
              "      <td>260</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5bba08db-2139-4423-b5f5-afad4d4d2a0a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5bba08db-2139-4423-b5f5-afad4d4d2a0a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5bba08db-2139-4423-b5f5-afad4d4d2a0a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                keyword  volume  topic\n",
              "0                 managing remote teams    1000      2\n",
              "1                          remote teams     390      2\n",
              "2  collaboration tools for remote teams     320      0\n",
              "3         online games for remote teams     320      1\n",
              "4            how to manage remote teams     260      2"
            ]
          },
          "execution_count": 41,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "NUM_TOPICS = 4\n",
        "kmeans = KMeans(n_clusters=NUM_TOPICS, random_state=21, n_init=\"auto\").fit(embeds)\n",
        "df['topic'] = list(kmeans.labels_)\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fLUzQoJD0IdY"
      },
      "source": [
        "### Generate Topic Names with Topically"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "WGVc0OewbShJ",
        "outputId": "2015f706-dfe7-416a-974e-daad98d072db"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-5637693d-1bb2-4fb0-828f-d698b5677d68\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>keyword</th>\n",
              "      <th>volume</th>\n",
              "      <th>topic</th>\n",
              "      <th>topic_name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>managing remote teams</td>\n",
              "      <td>1000</td>\n",
              "      <td>2</td>\n",
              "      <td>Managing remote teams</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>remote teams</td>\n",
              "      <td>390</td>\n",
              "      <td>2</td>\n",
              "      <td>Managing remote teams</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>collaboration tools for remote teams</td>\n",
              "      <td>320</td>\n",
              "      <td>0</td>\n",
              "      <td>Collaboration tools for remote teams</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>online games for remote teams</td>\n",
              "      <td>320</td>\n",
              "      <td>1</td>\n",
              "      <td>Virtual games for remote teams</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>how to manage remote teams</td>\n",
              "      <td>260</td>\n",
              "      <td>2</td>\n",
              "      <td>Managing remote teams</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5637693d-1bb2-4fb0-828f-d698b5677d68')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5637693d-1bb2-4fb0-828f-d698b5677d68 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5637693d-1bb2-4fb0-828f-d698b5677d68');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                keyword  volume  topic  \\\n",
              "0                 managing remote teams    1000      2   \n",
              "1                          remote teams     390      2   \n",
              "2  collaboration tools for remote teams     320      0   \n",
              "3         online games for remote teams     320      1   \n",
              "4            how to manage remote teams     260      2   \n",
              "\n",
              "                             topic_name  \n",
              "0                 Managing remote teams  \n",
              "1                 Managing remote teams  \n",
              "2  Collaboration tools for remote teams  \n",
              "3        Virtual games for remote teams  \n",
              "4                 Managing remote teams  "
            ]
          },
          "execution_count": 42,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Load topically\n",
        "app = Topically('api_key') # Add your Cohere API key here\n",
        "\n",
        "# Name clusters\n",
        "df['topic_name'], _ = app.name_topics((df['keyword'], df['topic']))\n",
        "\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        },
        "id": "_Ae0wTaW8HiO",
        "outputId": "69c73a1f-2308-40a0-b368-c8035b44f256"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collaboration tools for remote teams\n",
            "Virtual games for remote teams\n",
            "Managing remote teams\n",
            "remote team building activities\n"
          ]
        }
      ],
      "source": [
        "# View the list of topics\n",
        "topic2name = {}\n",
        "topic2name = dict(df.groupby('topic')['topic_name'].first())\n",
        "for key, value in topic2name.items():\n",
        "  print(value)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pMQzL5sL0YNN"
      },
      "source": [
        "# Step 3: Generate Blog Post Ideas for Each Topic"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1tlIS2db0fWU"
      },
      "source": [
        "### Take the Top Keywords from Each Topic"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "1bnYoTUCGQTv",
        "outputId": "cabdd711-641f-44e7-b7c5-19bf85f2512c"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "TOP_N = 10\n",
        "\n",
        "# Group the DataFrame by topic and select the top N keywords sorted by volume\n",
        "top_keywords = (df.groupby('topic')\n",
        "                        .apply(lambda x: x.nlargest(TOP_N, 'volume'))\n",
        "                        .reset_index(drop=True))\n",
        "\n",
        "\n",
        "# Convert the DataFrame to a nested dictionary\n",
        "content_by_topic = {}\n",
        "for topic, group in top_keywords.groupby('topic'):\n",
        "    keywords = ', '.join(list(group['keyword']))\n",
        "    topic2name = topic2name = dict(df.groupby('topic')['topic_name'].first())\n",
        "    topic_name = topic2name[topic]\n",
        "    content_by_topic[topic] = {'topic_name': topic_name, 'keywords': keywords}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 285
        },
        "id": "erNJQT3a5kkD",
        "outputId": "f8b0c153-0bc9-46bd-b7fe-e5d0283dc3ab"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "{0: {'topic_name': 'Collaboration tools for remote teams',\n",
              "  'keywords': 'collaboration tools for remote teams, best collaboration tools for remote teams, tools for remote teams, zapier remote teams, best communication tools for remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams, project management tools for remote teams, best tools for remote teams, collaboration remote teams'},\n",
              " 1: {'topic_name': 'Virtual games for remote teams',\n",
              "  'keywords': 'online games for remote teams, games for remote teams, retro ideas for remote teams, retrospective games for remote teams, virtual games for remote teams, agile games for remote teams, fun games for remote teams, whiteboard for remote teams, always on video for remote teams, best games for remote teams'},\n",
              " 2: {'topic_name': 'Managing remote teams',\n",
              "  'keywords': 'managing remote teams, remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, scrum remote teams, best practices for managing remote teams, manage remote teams, slack best practices for remote teams'},\n",
              " 3: {'topic_name': 'remote team building activities',\n",
              "  'keywords': 'team building activities for remote teams, team building for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams, communication exercises for remote teams, engagement activities for remote teams'}}"
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Print the topics and they top keywords\n",
        "content_by_topic"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCXY5cNj0qAW"
      },
      "source": [
        "### Create a Prompt with These Keywords"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "M-g_FiA6fX2w",
        "outputId": "d4545e7c-ca43-4b58-ac87-2f2c4227eadc"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def generate_blog_ideas(keywords):\n",
        "  prompt = f\"\"\"{keywords}\\n\\nThe above is a list of high-traffic keywords obtained from a keyword research tool. \n",
        "Suggest three blog post ideas that are highly relevant to these keywords. \n",
        "For each idea, write a one paragraph abstract about the topic. \n",
        "Use this format:\n",
        "Blog title: <text>\n",
        "Abstract: <text>\"\"\"\n",
        "  \n",
        "  response = co.generate(\n",
        "    model='command',\n",
        "    prompt = prompt,\n",
        "    max_tokens=300,\n",
        "    temperature=0.9)\n",
        "  return response.generations[0].text\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wTV6GG8i0tdK"
      },
      "source": [
        "### Generate Content Ideas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "-qGuVkIfmJ_d",
        "outputId": "5b7588cb-cfda-4590-80ca-532fceb6f125"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Topic Name: Collaboration tools for remote teams\n",
            "Top Keywords: collaboration tools for remote teams, best collaboration tools for remote teams, tools for remote teams, zapier remote teams, best communication tools for remote teams, free collaboration tools for remote teams, free retrospective tools for remote teams, project management tools for remote teams, best tools for remote teams, collaboration remote teams\n",
            "Blog Post Ideas: \n",
            "\n",
            "Blog title: The Best Collaboration Tools for Remote Teams\n",
            "Abstract: Are you looking for ways to improve collaboration among your remote team? In this blog post, we'll explore some of the best collaboration tools on the market and how they can help your team stay connected and productive, even when working from different locations. From video conferencing to project management tools, we'll cover everything you need to know to choose the right tools for your team.\n",
            "\n",
            "Blog title: Free Retrospective Tools for Remote Teams\n",
            "Abstract: Are you looking for ways to improve the effectiveness of your remote team? One way to do so is by conducting retrospectives, or meetings that allow team members to reflect on their past work and identify areas for improvement. In this blog post, we'll explore some of the best free retrospective tools on the market and how they can help your team stay focused and productive. From digital whiteboards to team chat apps, we'll cover everything you need to know to choose the right tools for your team.\n",
            "\n",
            "Blog title: Project Management Tools for Remote Teams\n",
            "Abstract: Are you looking for ways to improve the efficiency of your remote team? One way to do so is by using project management tools, which can help your team stay organized and on track. In this blog post, we'll explore some of the best project management tools on the market and how they can help your team meet deadlines and deliver high-quality work, even when working from different locations. From task management\n",
            "--------------------------------------------------\n",
            "Topic Name: Virtual games for remote teams\n",
            "Top Keywords: online games for remote teams, games for remote teams, retro ideas for remote teams, retrospective games for remote teams, virtual games for remote teams, agile games for remote teams, fun games for remote teams, whiteboard for remote teams, always on video for remote teams, best games for remote teams\n",
            "Blog Post Ideas: \n",
            "\n",
            "Blog title: The Best Online Games for Remote Teams\n",
            "Abstract: Looking for virtual games to play with your remote team? Check out this list of the best online games for remote teams, including whiteboard games, retrospective games, and fun games.\n",
            "\n",
            "Blog title: 5 Agile Games for Remote Teams\n",
            "Abstract: Looking for ways to inject some agile fun into your remote team's routine? Check out this list of 5 agile games for remote teams, including the always-on video game and the virtual whiteboard game.\n",
            "\n",
            "Blog title: The Dos and Don'ts of Virtual Team Retreats\n",
            "Abstract: Planning a virtual team retreat? Check out this list of dos and don'ts to ensure that your retreat is a success.\n",
            "--------------------------------------------------\n",
            "Topic Name: Managing remote teams\n",
            "Top Keywords: managing remote teams, remote teams, how to manage remote teams, leading remote teams, managing remote teams best practices, remote teams best practices, scrum remote teams, best practices for managing remote teams, manage remote teams, slack best practices for remote teams\n",
            "Blog Post Ideas: \n",
            "\n",
            "Blog title: The Secret to Managing Remote Teams\n",
            "Abstract: Are you tired of the common challenges faced by remote teams? Learn how to effectively manage your remote teams with these best practices. From setting clear expectations to establishing a communication cadence, you'll find helpful tips to improve your team's productivity and collaboration.\n",
            "\n",
            "Blog title: The Dos and Don'ts of Leading Remote Teams\n",
            "Abstract: So you've decided to lead a remote team. Now what? Avoid the common pitfalls of managing remote teams with these dos and don'ts. From setting clear goals to establishing a communication cadence, you'll find helpful tips to improve your team's productivity and collaboration.\n",
            "\n",
            "Blog title: How to Succeed as a Remote Scrum Team\n",
            "Abstract: Are you using the Scrum framework with a remote team? Discover the best practices for remote Scrum teams, including how to establish a virtual Scrum board, conduct effective daily standups, and maintain focus during sprint planning. With these tips, you'll be on your way to achieving success as a remote Scrum team.\n",
            "--------------------------------------------------\n",
            "Topic Name: remote team building activities\n",
            "Top Keywords: team building activities for remote teams, team building for remote teams, retrospective ideas for remote teams, team building ideas for remote teams, fun retrospective ideas for remote teams, team building exercises for remote teams, trust building exercises for remote teams, activities for remote teams, communication exercises for remote teams, engagement activities for remote teams\n",
            "Blog Post Ideas: \n",
            "\n",
            "Blog title: Team Building Activities for Remote Teams\n",
            "Abstract: Are you in charge of a remote team and looking for fun activities to help your team connect, communicate, and collaborate? This blog post will provide you with a list of remote-friendly team building activities and ideas that will help your team build trust, improve communication, and increase engagement.\n",
            "\n",
            "Blog title: Trust Building Exercises for Remote Teams\n",
            "Abstract: Are you having trouble building trust with your remote team? This blog post will provide you with a list of trust-building exercises and ideas that will help your team build trust, improve communication, and increase engagement.\n",
            "\n",
            "Blog title: Communication Exercises for Remote Teams\n",
            "Abstract: Are you having trouble communicating with your remote team? This blog post will provide you with a list of communication exercises and ideas that will help your team improve communication, increase engagement, and build trust.\n",
            "--------------------------------------------------\n"
          ]
        }
      ],
      "source": [
        "# Generate content ideas\n",
        "for key,value in content_by_topic.items():\n",
        "  value['ideas'] = generate_blog_ideas(value['keywords'])\n",
        "\n",
        "\n",
        "# Print the results\n",
        "for key,value in content_by_topic.items():\n",
        "  print(f\"Topic Name: {value['topic_name']}\")\n",
        "  print(f\"Top Keywords: {value['keywords']}\")\n",
        "  print(f\"Blog Post Ideas: {value['ideas']}\")\n",
        "  print(\"-\"*50)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Anr0Hu3hLxE3"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
